Vascular selection from images

ABSTRACT

Methods and systems for manually assisted definition of vascular features are described. In some embodiments, a method provides for editing of vascular paths by enabling a user to drag an erroneously segmented region of a selected vascular path into alignment with a more correctly segmented position that is depicted as a blood vessel in a vascular image. The method may use an energy function, defined as a function of position along the segmentation of the selected blood vessel, to determine how a vascular path is to be moved based on dragging motions provided by the user. In some instances, non-zero regions of the energy function are set based on the position of the selected region.

PRIORITY CLAIM

The present application is a continuation of U.S. patent applicationSer. No. 16/600,871, filed on Oct. 14, 2019, which is a divisionalapplication of U.S. patent application Ser. No. 15/959,024, filed onApr. 20, 2018, now U.S. Pat. No. 10,441,235, which is a continuationapplication of International Application No. PCT/IL2017/050544, filed onMay 16, 2017, which claims priority to U.S. Provisional PatentApplication No. 62/336,848, filed May 16, 2016, the entire contents ofeach of which are incorporated herein by reference and relied upon.

TECHNICAL FIELD

The present disclosure relates in general to the field of anatomicalsegmentation and more particularly, to manually assisted segmentation ofbranched vascular anatomy.

BACKGROUND

Vascular segmentation and feature identification is a preliminary stageof image-based measurement of vascular state. Though many stages ofvascular segmentation and feature identification can be performed basedprimarily on automated analysis, relevant image features are often oflow contrast and/or embedded in a complex environment comprisingelements of ambiguous geometry and extraneous features. Humansupervision may be introduced into the workflow to make corrections andhelp ensure quality of results, resulting in a semi-automated process,for example as in the Livewire and related procedures (discussed, forexample, in Ryan Dickie, et al.; Live-vessel: Interactive vascular imagesegmentation with simultaneous extraction of optimal medial and boundarypaths. Technical report TR 2009-23, School of Computing Science, SimonFraser University, Burnaby, BC, Canada, November 2009).

Additional background art includes:

an article titled: “Snakes: Active contour models”, by M. Kass, A.Witkin, and D. Terzopoulos, published in Int. J. Comput. Vis. (1987),1:321-331;

an article titled: “Multiscale vessel enhancement filtering”, by A. FFrangi, W. J. Niessen, K. L. Vincken, M. A. Viergever, published inMedical Image Computing and Computer-Assisted Intervention—MICCA'98; and

an article titled: “Snakes, Shapes, and Gradient Vector Flow”, by C. Xuand J. L. Prince, published in IEEE Transactions on Image Processing(1998), 7:359-369.

SUMMARY

There is provided, in accordance with some exemplary embodiments, amethod of segmenting a vascular image into vascular paths for definingpaths of blood flow. The method includes defining first and secondtargeted vascular path end regions within the vascular image,identifying the positions of vascular portions in the vascular image,and automatically generating a plurality of vascular path options fromthe identified vascular portions, each vascular path option defining apotential vascular path which extends between the first and secondtargeted vascular path end regions. The method may also includedisplaying the plurality of vascular path options registered to thevascular image for selection by a user, each of the displayed vascularpath options including the first and second targeted vascular path endregions. The example method may further include receiving a path optionselected by the user for defining a path of blood flow.

According to some embodiments, the plurality of paths are automaticallygenerated based on a first set of criteria, and the path option selectedby the user is selected based on a second set of criteria.

According to some embodiments, the predetermining comprises ranking thevascular path options in an order, based on assessment of a likelihoodthat each vascular path option corresponds to an actual path of bloodflow in blood vessels imaged in the vascular image.

According to some embodiments, the predetermining comprises applying acost function which assigns numerical costs to one or more featuresrelated to the vascular path options.

According to some embodiments, the cost function assigns numerical costsbased on features of a plurality of vascular segment centerlines fromwhich the vascular path option is concatenated.

According to some embodiments, the features of the plurality of vascularsegment centerlines include one or more from the group consisting ofcenterline orientation, centerline offset, and a count of centerlinesextending from a nodal region.

According to some embodiments, the cost function assigns numerical costsbased on features of the vascular image over which the vascular pathoption extends.

According to some embodiments, the features of the vascular imageinclude one or more of the group consisting of: continuity of vascularsegment image intensity, continuity of vascular segment image width, andthe position of a relative change in vascular intensity with respect toa nodal region from which three or more vascular segments extend.

According to some embodiments, the predetermining comprises applying acost function which assigns numerical costs based on an estimatedrelative position of a vascular segment image in depth, relative to anaxis extending perpendicular to a plane of the vascular image.

According to some embodiments, the presenting comprises presenting theplurality of vascular path options in a sequential order determined bythe order of selection.

According to some embodiments, the presenting comprises presenting theplurality of vascular path options simultaneously, and the order ofselection corresponds to an order in which the vascular path options aredisplayed as active for selection.

According to some embodiments, each vascular path option defines avascular path which extends through an image region between the firstand second targeted vascular path end regions, ending at a vascularregion of the image which is nearest to one of the first and secondtargeted vascular path end positions.

There is provided, in accordance with some exemplary embodiments, amethod of editing a vascular path to more accurately delineate asegmentation of a blood vessel in a vascular image. The example methodincludes receiving an indication of a selected region along thesegmentation of the blood vessel, defining an energy functional definedas a function of position along the segmentation of the blood vessel,wherein non-zero regions of the energy functional are set based on theposition of the selected region. The method may also include movingregions of the segmentation in accordance with energy minimizationwithin the non-zero regions of the energy functional.

According to some embodiments, energy functional values in the non-zeroregions are set based on features of the underlying vascular image.

According to some embodiments, energy functional values in the non-zeroregions are set based on movement of a user-controlled positionindication.

According to some embodiments, the user-controlled position indicationcomprises the indication of the selected region.

There is provided, in accordance with some exemplary embodiments, a userinterface for semi-automatic segmentation of a vascular path, the userinterface comprising: at least one interface module operable to presentan automatically generated to default vascular path extending betweentwo target end points; at least one interface module operable to presentat least one additional automatically generated vascular path extendingbetween the two target end points.

According to some embodiments, the user interface further comprises atleast one interface module allowing definition of at least one waypoint, and operable to present an automatically generated vascular pathextending between the two target end points via the at least one waypoint.

According to some embodiments, the user interface further comprises atleast one interface module operable to modify a previously definedvascular path by dragging a portion of the vascular path to a newlocation.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the disclosure pertains. Although a system, amethod, an apparatus, and/or a computer program product similar orequivalent to those described herein can be used in the practice ortesting of embodiments disclosed herein, exemplary systems, methods,apparatuses, and/or computer program products are described below. Incase of conflict, the patent specification, including definitions, willcontrol. In addition, the systems, methods, apparatuses, computerprogram products, and examples are illustrative only and are notintended to be necessarily limiting.

As will be appreciated by one skilled in the art, aspects of the presentdisclosure may be embodied as a system, a method or a computer programproduct. Accordingly, aspects of the present disclosure may take theform of an entirely hardware embodiment, an entirely software embodiment(including firmware, resident software, micro-code, etc.) or anembodiment combining software and hardware aspects that may allgenerally be referred to herein as a “circuit,” “module” or “system.”Furthermore, some embodiments of the present disclosure may take theform of a computer program product embodied in one or more computerreadable medium(s) having computer readable program code embodiedthereon. Implementation of the method and/or system of some embodimentsof the disclosure can involve performing and/or completing selectedtasks manually, automatically, or a combination thereof. Moreover,according to actual instrumentation and equipment of some embodiments ofthe method and/or system of the disclosure, several selected tasks couldbe implemented by hardware, by software or by firmware and/or by acombination thereof (e.g., using an operating system).

For example, hardware for performing selected tasks according to someembodiments of the disclosure could be implemented as a chip or acircuit. As software, selected tasks according to some embodiments ofthe disclosure could be implemented as a plurality of softwareinstructions executed by a computer using any suitable operating system.In an exemplary embodiment of the disclosure, one or more tasks,according to some exemplary embodiments of a method and/or a system asdescribed herein, are performed by a data processor, such as a computingplatform for executing a plurality of instructions. Optionally, the dataprocessor includes a volatile memory for storing instructions and/ordata and/or a non-volatile storage, for example, a magnetic hard-diskand/or removable media, for storing instructions and/or data.Optionally, a network connection may be provided. A display and/or auser input device such as a keyboard or mouse may also be provided.

Any combination of one or more computer readable medium(s) may beutilized for some embodiments of the disclosure. The computer readablemedium may be a computer readable signal medium or a computer readablestorage medium. A computer readable storage medium may be, for example,but not limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, or device, or any suitablecombination of the foregoing. More specific examples (a non-exhaustivelist) of the computer readable storage medium would include thefollowing: an electrical connection having one or more wires, a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a magnetic storage device,or any suitable combination of the foregoing. In the context of thisdocument, a computer readable storage medium may be any tangible mediumthat can contain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium and/or data usedthereby may be transmitted using any appropriate medium, including butnot limited to wireless, wireline, optical fiber cable, RF, etc., or anysuitable combination of the foregoing.

Computer program code for carrying out operations for some embodimentsof the present disclosure may be written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Java, Smalltalk, C++ or the like and conventionalprocedural programming languages, such as the “C” programming languageor similar programming languages. The program code may execute entirelyon the user's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Some embodiments of the present disclosure may be described below withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems) and computer program products according toembodiments of the disclosure. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatuses, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Additional features and advantages of the disclosed system, method, andapparatus are described in, and will be apparent from, the followingDetailed Description and the Figures.

BRIEF DESCRIPTION OF THE DRAWINGS

Some embodiments of the example systems, methods, apparatuses, and/orcomputer program products are herein described, by way of example only,with reference to the accompanying drawings. With specific reference nowto the drawings in detail, it is stressed that the particulars shown areby way of example, and for purposes of illustrative discussion ofembodiments of the systems, methods, apparatuses, and/or computerprogram products. In this regard, the description taken with thedrawings makes apparent to those skilled in the art how embodiments ofthe systems, methods, apparatuses, and/or computer program products maybe practiced.

In the drawings:

FIG. 1 is a schematic flowchart of a method for defining vascularcenterline paths on an angiographic image, according to some exemplaryembodiments of the present disclosure;

FIG. 2 is a schematic flowchart of a method for generating vascular pathoptions from a vascular image, according to some exemplary embodimentsof the present disclosure;

FIG. 3A is a schematic flowchart of a method for selecting and/ordefining a particular vascular path option, according to some exemplaryembodiments of the present disclosure;

FIG. 3B is a schematic flowchart of a method for manually editing avascular path, according to some exemplary embodiments of the presentdisclosure;

FIGS. 4A-4F schematically illustrate selection of vascular path options,according to some exemplary embodiments of the present disclosure;

FIGS. 5A-5C schematically illustrate selection from among alternativevascular path options, according to some exemplary embodiments of thepresent disclosure;

FIGS. 5D-5E schematically illustrate a method of manually defining avascular path option, according to some exemplary embodiments of thepresent disclosure;

FIG. 5F schematically illustrates manual editing of a vascular pathoption, according to some exemplary embodiments of the presentdisclosure;

FIG. 6 schematically illustrates a display of path definitions madeconcurrently on a plurality of different vascular image views, accordingto some exemplary embodiments of the present disclosure; and

FIG. 7 is a schematic diagram of software modules and data structuresimplemented in a system for semi-automated segmentation of vascularpaths, according to some exemplary embodiments of the presentdisclosure.

DETAILED DESCRIPTION

The present disclosure, in some embodiments thereof, relates to thefield of anatomical segmentation and more particularly, to manuallyassisted segmentation of branched vascular anatomy.

Overview

A broad aspect of some embodiments of the current disclosure relates tomethods for combining manual and automatic segmentation techniques,which potentially allows for efficient, reliable vascular treegeneration.

Vascular segmentation is an early step in the characterization ofvascular anatomy and function from medical image data for a range ofapplications, including studies of blood flow. However, automaticsegmentation methods typically display degraded performance asrequirements become more stringent. In addition, contrast agents arelimited in their usable concentration; injectable contrast agentsquickly dilute, limiting available imaging time. Further, for safetyreasons, radiation doses used in some types of imaging are preferablykept to a minimum necessary for reliable visualization. Even though highcontrast and/or high signal-to-noise can often be achieved for largeblood vessels, there is also a need in some applications to analyzesmaller vessels. However, imaging quality rapidly degrades withdecreased vascular diameter, as the signal intensity approaches thelimit of background noise, or even quantum noise inherent to the signalitself. Apart from these technical considerations, vasculature itself ishighly complicated in form. This potentially gives rise, particularly in2-D images, to many cases of ambiguous structure; difficult to resolveto a particular branch structure by inspection at the level of localfeatures. Global feature detection, on the other hand, is difficult toaddress by general automatic methods, as interpreting such featurespotentially relies on particularities of the constraints applicable fora particular structure and/or imaging method.

For these and other reasons, the practical case often results that thequality of medical images available for analysis is, at least for somevascular structures of interest, near or beyond the limits of presenttechniques for machine vision and/or image processing. Even if theboundaries of these limits should shift over time as technologydevelops, it is to be expected that there will continue to beinteresting segmentation problems for which automatic vascularsegmentation alone is inadequate.

Semi-automatic segmentation methods seek to address limitations ofpurely automatic segmentation methods by augmenting them with humanjudgement and/or control. However, human intervention is expensive interms of time, money, and/or availability. In view of this, a goal insome semi-automatic segmentation methods is to reduce human time and/oreffort spent in supervising automatic segmentation. Considered as aproblem in optimization, the goal, in some embodiments of thedisclosure, may be to bring human intervention in semi-automaticsegmentation to the lowest achievable level which is consistent withresults of sufficiently quality for the use to which they are applied.Accordingly, preferred methods for semi-automatic segmentation mayinclude features which are specific to application in particular problemdomains.

An aspect of some embodiments of the present disclosure relates tocascading methods for semi-automatic vascular segmentation ofangiographic images. More particularly, in some embodiments, the aspectrelates to cascading methods for semi-automatic segmentation ofangiographic images while working within the constraints of producingresults in real-time. Optionally, segmentation is completed while acatheter procedure, which may be been used to produce the images, isstill underway, while leaving sufficient time for subsequent analysis(e.g., analysis leading to diagnosis and/or treatment planning).

In some embodiments, the manually supervised operations of thesemi-automatic vascular segmentation are structured to cascade throughan increasingly attention-demanding set of user operations, where thecascading is stopped once the user is satisfied that a sufficientquality of result has been obtained. There may be one or more routesthrough this cascade of operations; for example, the order of operationschosen optionally depends on whether nearly adequate results areobtained early that only need minor editing, or whether a suitable pathneeds to be defined by a user de novo. In some embodiments, the cascadeis structured so that more likely options are presented earlier and/orwith greater emphasis, potentially reducing time and/or effort spent bya user in making selections. Optionally, the order of operations isselected to emphasize getting “close enough” results with minimum input,while also providing an opportunity for the correction of errors inautomatically identified results as necessary.

In some embodiments, the method optionally starts with the definition oftwo vascular path end regions (e.g., regions within some maximumdistance from a selection point); after which a most-likely andautomatically detected path is presented to the user for acceptance orrejection. For a suitable definition of “most-likely” (e.g., a suitablecost function), this potentially allows most or a plurality of userinterventions to be limited to simple acceptance of a default.Optionally a plurality of vascular path end region pairs are definedinitially, and a corresponding plurality of default options arepresented simultaneously, and user interaction is limited still furtherto correcting defaults. In some embodiments, definition of one of theendpoints is simplified by defining at least one endpoint at a rootposition of the vascular tree, positioned such that it may be consideredto be at one end of any vascular path leading back from one of itsbranches. In some embodiments, definition of a plurality of vascularterminal positions is based on fully automatic detection (e.g.,positions where a vascular skeleton defining vascular centerlinesnaturally ends), or a semi-automatic method such as positions where aselection line swept out by the user intersects segments of anautomatically detected vascular skeletonization.

If a default path is not accepted, in some embodiments, thendecreasingly likely (higher cost function scored), automaticallysuggested paths are optionally presented as available. For example, theuser can use a scroll wheel or other control to quickly show alternativepaths that extend between some pair of target vascular path end regions.

Failing to find an adequate path among automatically suggestedalternatives, in some embodiments, the user is provided with a userinterface tool which is operable to define a vascular path based on thedefinition of one or more additional waypoints.

Additionally or alternatively, one or more editing tools are optionallyprovided, which allow a nearly-acceptable presented alternative to bemodified. For example, a path may be cut, extended, and/or merged with aportion of another existing path. In some embodiments, a path isoptionally edited along its length, for example, by re-tracing,redefinition of anchor points, and/or dragging of erroneously segmentsregions into their correct position.

In some embodiments of the disclosure, a target of the semi-automaticvascular segmentation is the production of one or more vascular pathscorresponding to anatomically valid paths of blood flow. In someembodiments, a vascular path comprises a numerically stored sequence ofpositions corresponding to vascular image positions. In someembodiments, the vascular path comprises a vascular center-line.Optionally, the vascular path is defined at one end by a root position,located at the path end, which is for example, within the least-branchedvessel the path traverses. At the other end, the vascular path isoptionally defined by a terminal position, which is, for example,located within the most-branched vessel the path traverses.

In some embodiments, vascular paths are defined by the concatenation ofone or more vascular segments. Optionally, vascular segments are definedby one or more methods, at one or more levels of fidelity. In someembodiments, a vascular segment may be defined within a skeletonizedrepresentation of a vascular image (e.g., a binary pixel-arrayrepresentation which extends through the detected extent of thevasculature with a one-pixel width). Such a vascular segment optionallycomprises a sequence of pixel locations extending between two pixelswhich mark its end points. The end points are optionally selected by anyconvenient method, even arbitrarily (e.g., by breaking the skeleton intosegments of at most N pixels in length). Preferably, however, vascularsegment end points are defined at branches and/or crossings (e.g.,skeleton pixels from which branches lead in at least three directions),and/or at free ends (e.g., skeleton pixels from which only one branchleads).

In some embodiments, vascular paths are defined separately from oneanother. In some embodiments, vascular paths are defined by theirdifferent extents along a branched vascular tree (e.g., defined by aparticular path of traversing the branches of vascular tree; optionallya vascular tree defined as a set of linked vascular segments). In someembodiments, in contrast, a vascular tree is defined by the merger of aset of vascular paths (e.g., paths which share a common vascular segmentare also considered to share a common root segment).

An aspect of some embodiments of the present disclosure relates tomethods of selecting vascular paths based on the ordered presentation ofautomatically generated vascular path options.

In some embodiments of the disclosure, a plurality of path routesextending along detected vascular segments (e.g., vascular segmentsdefined according to criteria of gradient, curvature, and/or relativeintensity) are generated for a pair of end regions. Optionally, thegenerated path routes are those which reach to segment points which arewithin some region; optionally the region is defined, for example, asthe region within some maximum distance from an end-point. Thisdistinction is relevant, for example, in case one of the end points isnot itself on a segment.

In some embodiments, the plurality of path routes is presented as arange of vascular path options in a pre-determined order. Optionally,the pre-determined order is based on a cost function constructed to rankpath routes according to criteria (optionally, heuristic criteria) thatassign more-likely actual paths of blood flow between two end points alower cost value than less-likely actual paths of blood flow.Preferably, path routes are presented along with image data from whichthey derive, for example, as graphical overlays on the image. In someembodiments, image data is presented as an animated sequence of images(e.g., images between which the vasculature moves slightly, and/or isviewed from a different angle). Potentially, these differences harnessvisual capabilities more particular to the user than to the automaticdetection algorithm, for example, by emphasizing connectedness amongportions of the vasculature which move together.

In some embodiments, cost function criteria include aspects of vascularcontinuity, vascular branching anatomy, criteria based on thethree-dimensional shape of an organ which the vasculature perfuses,vascular image opacity, and/or other criteria.

In some embodiments, automatic segmentation results potentially do notunambiguously distinguish which path, among a plurality of alternativevascular paths between two vascular positions, corresponds to an actualpath of blood flow. Particularly in the case of 2-D images of 3-Dvascular structures (e.g., vascular structure that is typicallyvisualized extending through two or more layers of depth) branches ofthe same vascular tree may appear to cross over each other. Althoughsame-tree crossover is seldom observed in structures which are typicallyimaged within a single focal plane, such as inner retinal vasculature,ambiguities may arise in cases where distinguishing between venous andarterial structures is difficult.

It is noted that dynamic path extending methods, such as Livewire,partially mitigate automatic segmentation ambiguities by allowing thedefinition of waypoints that can be set to force the automatic resultinto the correct segmentation. However, this is potentially moretime-consuming than simply defining start and end points. In someembodiments, there is a tradeoff between a certain but slow vascularpath definition by use of waypoints, and a less certain but potentiallyfaster vascular path definition by selection from among automaticallypredefined vascular path options.

An aspect of some embodiments of the present disclosure relates tomethods of editing vascular paths by dragging an erroneously segmentedregion into alignment with a more correctly segmented position.

In some embodiments, the method comprises receiving an indication of aselected region along the segmentation of the blood vessel. In someembodiments, the indication comprises a screen coordinate pointselection, for example, by pressing a button in conjunction with aparticular position of a screen cursor, and/or a gesture input to atouch screen. In some embodiments, the indication comprises directlyselecting an erroneously segmented region.

In some embodiments, the method comprises assigning motionsusceptibility characteristics to a segmentation path region surroundingthe selected region that enable it to be moved, preferably whilemaintaining the remainder of the segmentation in a state which is notsusceptible to movement. In some embodiments, the assigning ofsusceptibility to motion comprises defining an energy functional definedas a function of position along the segmentation of the blood vessel.Optionally, non-zero regions of the energy functional are set based onthe position of the selected region. In some embodiments, the methodcomprises moving regions of the segmentation in accordance with energyminimization within the non-zero regions of the energy functional.

Optionally, motion of the motion-susceptible portion of the segmentationpath region is in coordination with movement of a cursor. Optionally,motion of the motion-susceptible portion is at least partially governedby values and/or gradients of image intensity in vicinity of theselected region, such that the selected region behaves as thoughattracted to vascular regions as it moves.

Before explaining at least one embodiment of the disclosure in detail,it is to be understood that the example systems, methods, apparatuses,and/or computer program products are not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings. The example systems, methods, apparatuses,and/or computer program products are capable of other embodiments or ofbeing practiced or carried out in various ways.

Method of Defining Paths Along a Vascular Tree

Reference is now made to FIG. 1, which is a schematic flowchart of amethod of defining vascular centerline paths on an angiographic image,according to some exemplary embodiments of the present disclosure.

In some embodiments, an editing mode of a computer program configuredfor interactive vascular path definition via user interaction through auser interface is activated. At block 102, in some embodiments, a rootposition (corresponding, for example, to root positions 401, 501 ofFIGS. 4A-4F and 5A-5F) is defined. In some embodiments, the rootposition is the vascular position visible in the image which istopologically nearest to the region where blood enters or exits theheart. In the coronary arteries, for example, the root position istopologically nearest to the region where blood exits the left ventricleinto the aorta. Optionally, the root position is manually defined, forexample, by clicking on or touching (via a user interface device) apoint in the image; and/or by selecting, moving, and/or confirming anautomatically defined root position. Optionally, automatic detection ofone or more candidates for the root position is performed, for example,based on the timing and/or location of dye appearance in an image timeseries immediately after injection; and/or based on morphologicalcriteria such as vascular thickness, branch order and/or orientation,etc.

At block 110, in some embodiments, the defined root position is used asan input for the definition of path data including listing ofalternative path options and path option costs, for example as describedin relation to FIG. 2.

At block 112, in some embodiments, a vascular path representing a pathof continuous vascular connection between some point in a vascular treeimage and the root position is optionally defined (and/or selected, forexample, based on the listing of path alternative path options and pathoption costs defined at block 110). Examples of the implementation ofdetails of block 112 are described in relation to FIG. 3A.

At block 114, a vascular path representing a previously defined path ofcontinuous vascular connection between some point in a vascular treeimage and the root position is optionally edited. Examples of theimplementation of details of block 114 are described in relation to FIG.3B.

At block 104, in some embodiments, a determination is made as to whetheror not edit mode is continuing; if not the flowchart exits. Otherwise,flow returns to before block 112 (optionally, to before block 102, toallow redefinition of the root position).

Vascular Paths and Cost Functions

Reference is now made to FIG. 2, which is a schematic flowchart of amethod of generating vascular path options from a vascular image,according to some exemplary embodiments of the present disclosure.

The flowchart begins; and at block 208, in some embodiments, a vascularskeleton graph 206 is split into branches. In some embodiments, thevascular skeleton graph is derived from image processing of anangiographic image. Images 400 of FIGS. 4A-4F and 500 of FIGS. 5A-5F areexamples of the source images from which the vascular skeleton graph isderived. In some embodiments, the vascular skeleton graph is comprisedof vascular centerlines. An example of the extraction of vascularcenterlines (which in some embodiments uses anisotropic diffusion,Frangi filtering, and hysteresis thresholding, followed by binarythinning) is described, for example, in relation to block 20 of FIG. 14of International Patent Publication No. WO2014/111927 to the applicant,filed on Jan. 15, 2014; the contents of which are incorporated herein byreference in their entirety. In some embodiments, the basis of themethod used is similar to that introduced by Weickert in “A Scheme forCoherence-Enhancing Diffusion Filtering with Optimized RotationInvariance” and/or “Anisotropic Diffusion in Image Processing” (Thesis1996).

At block 208, in some embodiments, the vascular skeleton graph 206 issplit into branches. Optionally, this comprises noting pixels at whichthree or more skeleton segments converge (e.g., pixels with three ormore neighbors; optionally, short spurs are excluded fromconsideration), and assigning these points to be path breaks.Optionally, it is also noted which branches connect to which (connectionmay be merely apparent, as the branches may also cross one anotherand/or approach closely enough to appear to merge). The result of theoperations of block 208 is branch list 210. It should be noted that the“branches” of branch list 210 are potentially linked into loops, eitherdue to the actual underlying anatomy (for example, the development ofshunting vessels), and/or due to apparent connection arising fromvascular near approaches and/or crossings at the provided angle and/orresolution of the image.

In some embodiments, branches preserved in the branch list are onlythose for which a continuously connected route exists to a root position(e.g., root positions 401, 501 of FIGS. 4A-4F and 5A-5F) on the vasculartree. Optionally, the root position is determined, for example, asdescribed in relation to block 102 of FIG. 1.

At block 212, in some embodiments, paths leading from each branch to thedefined root position are calculated, for example, by use of a searchalgorithm. Optionally, there is more than one such path available, forexample due to crossings, close approaches, or other ambiguities presentin the image, as previously mentioned. Paths that internally include thesame vascular segment more than once are optionally excluded from thislist (to avoid paths that loop and/or double back on themselves). Thedata structure resulting from the operations of block 212 comprises pathlist 214. In some embodiments, path list 214 comprises each non-excludedpath identified that extends between the root position and each vascularskeleton segment.

At block 216, in some embodiments, path costs for each path in path list214 are calculated to produce a path cost list 218. Path cost list 218,in some embodiments, is structured so that members of a potentialplurality of paths in path list 214 reaching from any vascular segmentto the root can be rank ordered for likelihood of being an actual pathof blood flow. Use of path list 214 and/or path cost list 218 isdescribed further in connection with operations of block 112 of FIG. 1,for example as detailed in relation to FIG. 3A.

According to a convention adopted herein, higher cost paths areoptionally considered less likely as candidates to be an actual path ofblood flow in the vascular anatomy. Path cost criteria are optionallyevaluated, for example, to receiving a binary value (0 or 1, forexample), a ranked value, and/or a score on a continuous scale. Pathcost criteria optionally add, multiply, or are otherwise combined intoan overall cost function.

In some embodiments, results of machine learning are used to assigncriteria cost values, and/or contribution to overall cost relative toother criteria. In some embodiments, a machine learning techniquecomprises one or more implementations of decision tree learning,association rule learning, an artificial neural network, inductive logicprogramming, a support vector machine, cluster analysis, Bayesiannetworks, reinforcement learning, representation learning, similarityand metric learning, and/or a technique related to machine learning. Themachine learning is optionally based on a set of angiographic imageswhich have been separately marked for vascular tree morphology, and/orbased on results of self-learning.

Cost Criteria

In some embodiments, path costs recorded in the path cost list 218 arecalculated with respect to one or more of the following considerationsand/or criteria.

Node Type Classification

Herein, vascular skeleton segment connection points are referred to asnodes. Nodes may be defined as single pixels, or optionally, as regionsof larger size (e.g., a radius of 2, 3, 4, 5 or more pixels) withinwhich three or more vascular skeleton segments meet. In someembodiments, the overall cost function which is applicable to a nodedepends at least in part on a node type classification.

Where two vascular segments happen to cross in an image (e.g. due tobeing in different planes of the heart image), the vascular skeletonwill show four branches arising from approximately the region of acentral node. The corresponding vascular skeleton is optionally analyzedas defining a crossing-type node. In some embodiments, where a bloodvessel branches, bifurcation is typical. In such cases, the node isoptionally defined as a junction node from which three vascular skeletonbranches arise. Cost function criteria applicable to these two basicnode types are optionally differently assigned.

In some embodiments, there are other junction orders which can appear inthe path list. For example, a bifurcation may happen to be at the sameposition as a vascular crossing point. Another situation whichpotentially arises is where two junction nodes appear so close to oneanother that they could also be analyzed as four-branched node.Contrariwise, skeletonization artifacts potentially introduce a slightoffset to the same blood vessel on either side of a crossing, making itappear more like two adjacent junction-type nodes. Three- andfour-branched nodes can also arise where a terminal vascular segment(that is, a vascular segment which is the last segment visible in theimage on its branch) happens to have a free end located at or near theposition of another vascular segment. Optionally, nodes comprising five,six, or more segments are analyzed as being composed of a suitablecombination of junction nodes, crossing nodes, and/or free-endterminations.

In some embodiments, morphological criteria besides branch counts areused in node classification for purposes of cost function assignment.For example, some close sequential bifurcations comprise just onecontinuously oriented pair of segments, while the other two segments runat a relative sharp angle (e.g., about 90°) relative to one another. Inanother example, bifurcation angles typically (though not exclusively)display forward-direction (in the direction of arterial flow) acuteangles. Backward-branching oblique angles are more likely to signify acrossing-type node. In yet another example, regions of vascularcrossings may appear as regions of increased radiopacity (due to thecumulative contribution of two overlapping vascular thicknesses),compared to regions of vascular branching. On the other hand, suddenvariations in apparent radiopacity on either side of anode potentiallyindicate a branching-type node, due to changes in vascular orientation(and corresponding changes in absorbing path length) relative to an axisperpendicular to the plane of the image.

Optionally, for example, for classification of more complex nodes,Murray's principle is applied, wherein r_(p) ³=r_(d) ₁ ³+r_(d) ₂ ³r_(d)₃ ³+ . . . +r_(d) _(n) ³; with r_(p) ³ being the radius of the parent(trunk) branch, and r_(d) ₁ ³,r_(d) ₂ ³,r_(d) ₃ ³ . . . r_(d) _(n) ³being the radii of the respective child branches. Optionally, a vascularsegment combination which satisfies Murray's principle at some node isassigned a higher probability of representing a branching-type node,with remaining segments assigned higher probabilities of being free-endterminations, or participants in crossing-type node structures.

In some embodiments, the appropriate cost function for the node isdependent on which analysis of the node-type is correct. In someembodiments, node-type determination is subject to probabilisticassignment (e.g., a node is assigned an 80% chance of being a crossingof two vessels, a 15% chance of being a double bifurcation (ortrifurcation), and a 5% chance of being a bifurcation in the region ofthe free end of a terminal vascular segment).

In some embodiments, there is no explicit classification made in thecost function between junction-type and crossing-type nodes (or freeends, or combinations of any of these types); rather, heuristics areadopted which operate on the basic relative vascular morphologies assuch. This approach may be suitable, for example, for certain types ofmachine learning-based implementations like artificial neural networks.Nevertheless, node type provides a convenient organizing concept forpurposes of explaining further vascular path cost function criteria.

Continuity and/or Consistency at Crossing-Type Nodes

In some embodiments of the disclosure, continuity of vascular morphologyat nodes is used as a basis of cost function value assignment.

The convergence of four vascular skeleton branches considered as apotential crossing-type node is optionally analyzed as defining an apriori choice of three directions to continue in (exit-side) from somefourth direction of approach (entrance-side). In some embodiments, oneor more criteria relating to continuity of morphology are applied todetermine which exit-side branch is the branch of most likelycontinuation from a given entrance-side.

One such criterion is vascular radius, wherein the exit-side vascularbranch, which is most similar in radius to the entrance-side, receivesthe correspondingly lowest cost assignment. Optionally, vascular radiusis measured as half the apparent width of a filled lumen in the image.Optionally continuity of image intensity values (which is partially, butnot only, a function of radius) is also taken into account (e.g., bydirect comparison of intensity values). Optionally, principles ofdensitometry are applied: for example, the blood vessel is modeled ascylinder of a certain radius filled with a substance having a particularconcentration and/or coefficient of absorption; and the cost function isbased jointly on continuity of radius and absorption properties thatsatisfy observed intensity values in the image.

In some embodiments, continuity of direction is a criterion: theexit-side vascular segment that is most similar in direction to theentrance-side segment optionally receives the correspondingly lowestcost assignment as its continuation. It is noted that densitometrydifferences can also arise based on the orientation of blood vessels inthe direction perpendicular to the plane of the image (e.g., a bloodvessel appears darker when seen more nearly end-on). Thus, continuity ofvascular intensity values is optionally a proxy for continuity ofvascular direction in this dimension.

In some embodiments, these criteria include one or more furtherrefinements. For example, a criterion of direction applied to a junctionmay optionally comprise a cost assignment based on the continuing rateand/or direction of angle change before and after a node (e.g.,continuation between two segments having a more similar radius ofcurvature receives a lower cost assignment). This is potentially ofparticular relevance for vascular segments that are rounding a bulge ofthe 3-D heart surface through a portion of the vascular image. In someembodiments, a criterion of the cost function favors paths along whichvascular width monotonically changes (e.g., increasing in the directionof the root).

In some embodiments, paths from segments A, B, C, D meeting at ajunction (particularly a crossing-type junction) are treated as“entangled”, such that a low cost for a path leading from A→Bcorresponding lowers the cost of the path C→D relative to alternativepath C→B. Optionally, a definitive indication by a user during somestage of vascular tree drawing that path A→B is a preferred path ofvascular flow is similarly entangled to lowering the cost assignment ofpath C→D compared to C→B.

Continuity and/or Consistency at Branching-Type Nodes

In some embodiments, where a blood vessel branches, bifurcation istypical. Optionally, a greater branch number, such as trifurcation, istreated as being composed of adjacent bifurcations. In a bifurcation,there is optionally defined a junction node at which there is an apriori choice of two directions to continue in (exit-side) from anythird direction of approach (entrance-side). In some embodiments, theheuristic is adopted that the exit-side branch which requires theminimal change in direction through the junction node receives the lowercost assignment.

Optionally, cost scoring based on vascular radius similarity is reducedin its influence on the cost function (e.g., by lower weighting) when athree-way junction node is detected, to account for two thin branchesbeing potentially nearer in size to each other than their mutual trunk.Additionally or alternatively, radius-based cost functions are assignedin a different way for branching-type nodes (e.g., the thickest vascularsegment (which generally lies in the direction of the root position) isgiven the lowest cost score for both of the two thinner branches).Optionally, a cost score assignment is more particularly based onMurray's principle, wherein paths that better satisfy this theoreticalrelationship are given correspondingly lower cost scores.

Other Criteria

In some embodiments, paths using vascular segments which run inrelatively straight and/or continuously arcing lines are optionallyscored with lower costs. This is a potential advantage for favoringimage features that are “more vascular” in character, as opposed toimage features which might end up in the vascular skeleton, but areactually non-vascular in origin. Cost is optionally calculated, forexample, based on the total area under the curve (above or below zero)of the derivative of the angle of local vascular segment orientation.

In some embodiments, vascular position in three dimensions is estimated.Optionally, use of this information is part of node classification,and/or used in continuity/consistency cost scoring. In some embodiments,a vascular tree is known a priori to extend across a three-dimensionallycurved surface, such as a surface defined by the shape of the heartmuscle. In some embodiments, parameters of this curved surface arederived from an image shadow delineating boundaries of the shape beingfollowed. In some embodiments, shapes and/or intensities of the vesselsthemselves identify shape boundaries. For example, as blood vesselscurve to be oriented more inwardly/outwardly from the plane of theimage, they potentially begin to appear more radiopaque at some boundarylimit as they present a longer absorbing cross-section. Vessel roundinga curve in the third dimension may appear to first approach, and thenbend away from, some boundary limit in the 2-D image. In someembodiments, a shell is modeled as comprising two portions, generallyoccupying different depths, but joined together at such a boundarylimit. Optionally, vascular segments are scored for their likelyposition on the overlying or underlying portion of the shell, whiletreating the boundary limit as a transition zone. Paths which requireabrupt transitions between two different shell portions (particularlyaway from the boundary limit), are optionally given a higher cost value.

Contrariwise, it is noted that vessels of the same type (artery or vein)which occupy the same anatomical plane may be assumed, in someembodiments, to be relatively unlikely to cross over one another.Optionally, where a crossing-type node is identified with highprobability (additionally or alternatively, where consistency andcontinuity suggests a vascular crossing with relatively low associatedpath cost), this is also used to increase the estimated likelihood thatthe two different vessels occupy two separated shell portions. Thenother vascular segments which directly contact them are alsocorrespondingly more likely to be in the same shell portion, informationwhich is optionally used to disambiguate assignment of path costs acrossnearby nodes.

Vascular Path Selection and/or Definition

Reference is now made to FIG. 3A, which is a schematic flowchart of amethod for selecting and/or defining a particular vascular path option,according to some exemplary embodiments of the present disclosure. Insome embodiments, the operations of FIG. 3A correspond to operationsoccurring within block 112 of FIG. 1. Reference is also made to FIGS.4A-4F, which schematically illustrate selection of vascular pathoptions, according to some exemplary embodiments of the presentdisclosure. FIG. 4A shows an angiographic image 400 illustrating aportion of a vascular tree 402 (a cardiac vasculature, in the example),including a currently defined root position 401. Contrast of theangiographic image 400 is reduced in FIGS. 4B-4F to make other elementsmore visible.

In some embodiments of the disclosure, there are provided one or moremanually guided methods of confirming, selecting, and/or definingvascular tree topology. In this context, there is introduced an“accepted path list” 324. Accepted path list 324 optionally includespaths selected from path list 214, edited paths based on paths in pathlist 214, and/or paths generated de novo, for example, on the basis ofuser input. In some embodiments, path cost list 214 is used indetermining default selections from path list 214, and/or an orderpresentation of additional selections from path list 214, as needed.

In connection with block 212 of FIG. 2 herein, the generation of a datastructure comprising a path list 214 is described. In some embodiments,path list 214 potentially comprises a number of vascular paths which are“true” vascular paths in the sense that they are isomorphic with avascular path that blood follows in the actual patient anatomy inflowing between the root position 401 and the distal end of the vascularpath. However, there may also be a number of “false” vascular paths inthe same path list 214; reflecting ambiguities introduced, for example,by vascular crossing points and/or limitations in vascular imagequality. Although path cost list 218 optionally provides a basis forpreferring some vascular paths to others, there may nevertheless becases where circumstances lead to the true vascular path being lesspreferred. In some embodiments, the true vascular path may not even beavailable in path list 214; for example, due to contrast dropouts and/orother artifacts which may affect the vascular skeletonization.

At block 310, in some embodiments, a path terminus is indicated (e.g.,by a user). In some embodiments, indication of the path terminuscomprises a “hover” cursor input event (e.g., cursor 405 moved into thevicinity of a potential vascular path, and paused long enough to bedetected for processing; a long touch on a touch screen, etc.). In someembodiments, another input event indicates the path terminus: forexample, a screen tap. In some embodiments, the user indication isreferred to a region (e.g., a pixel) on the closest available segmentwhich is represented in path list 214. Optionally, the user indicationis referred to the region of the segment which is closest to theposition of the indication. For example, vascular path 407 of FIG. 4Bextends to a path terminus 407A in the vicinity of cursor 405 at oneposition. When cursor 405 is moved as shown in FIG. 4C, a differentvascular path comprising vascular path 408 appended to vascular path 407is shown instead, again terminating at path terminus 408A in thevicinity of cursor 405. Optionally, the user indication is referred tothe terminal of the segment which is furthest from the root position401.

For purposes of explanation, the conversion of an indicated pathterminus to a path in the accepted path list 324 is primarily presentedin terms of manual operations relating to one path at a time. However,it is to be understood that these descriptions also apply, changed asnecessary, to the processing of multiple and/or automatically selectedpaths.

For example, in some embodiments, one or more candidate path termini aredetected automatically (for example, at the free ends of vascularskeleton segments which are in continuous connection with the rootposition 401). Additionally or alternatively, a plurality of pathtermini are optionally indicated by a user as part of a single inputaction (e.g., by dragging out a path (by cursor movement, touch input,or otherwise) which crosses among several candidate path termini). Theindicated termini are taken as the positions at which the dragged outpath intersects segments of the vascular skeleton.

When a plurality of candidate path termini are available, a user isoptionally presented with the option to step through candidate pathtermini (e.g., by a sequence of key presses, finger drags across a touchinput screen, scroll wheel movements, or another input), and a selectedterminus becomes an indicated path terminus. Optionally, initial pathsare defined (and optionally accepted as members of the accepted pathlist by default) for each of the plurality of candidate path termini.

Upon a first entry into block 311, in some embodiments, the lowest costavailable path (based on path cost list 218) from path list 214 that isamong those reaching to the indicated path terminus is initiallyselected for presentation to the user. Optionally, the presentation isdifferentiated as being active for current definition; for examplehighlighted with special coloration and/or thickness. It should be notedthat although paths from path list 214 are optionally the basis of whatis presented, in some embodiments, the actual path shown is derived froman entry in path list 214 by application of an active contour or dynamic“snake” algorithm, for example as described herein.

At block 312, in some embodiments, a user determines if the correct path(suitable for use as a “true” vascular path) has been displayed or not.If yes, then the flowchart optionally proceeds with the user issuing aconfirmation input (for example, a double click, screen tap, key press,or other input) at block 316. The currently selected path is then addedto the accepted path list 324, and the flowchart exits block 112 ofFIG. 1. Optionally, accepted paths remain displayed; optionallyindicated as deselected, for example, by being drawn thinner, with lowercontrast, or by another visual indication. FIG. 4D shows theconcatenation of paths 407 and 408 drawn as a member of the acceptedpath list 324. For the new position of cursor 405, a corresponding newpath option 409 is shown. This sequence continues in FIGS. 4E-4F, inwhich new paths 411 and 414 are created. As presented, each new path isdrawn as extending all the way back to root position 401. Alternatively,the display is extended only back as far as the first contact with apreviously accepted vascular path. Optionally (and optionallyindependent of display method), paths are stored in accepted path list324 as complete paths extending between terminus and root position, asincremental additions to a vascular tree structure, and/or in anotherformat.

Otherwise, if in block 312 the correct path has not been displayed, theflowchart continues; optionally to an option selection event at block314. In some embodiments, the option selection event comprises movementof a scroll wheel, a touch screen gesture (for example, a slidingmotion), a key press, or another user input event. Optionally, thisselection is interpreted in software as indicating that the current pathdisplay should move to another candidate (e.g. the next candidateavailable in order of cost as defined in path cost list 218). In avariant of the method, a plurality of vascular path candidates arepresented simultaneously for a single pair of end points, and theselection is used to indicate which candidate should be drawn asactively selected. A potential advantage of this variant is to allowmore rapid realization that no suitable candidate is available, and/orto allow simultaneous comparison of available options.

At block 316, in some embodiments, the user may decide that nopre-defined path of path list 214 is suitable for the currentlyindicated path terminus, and proceed to block 320 in order to define anew path manually. Otherwise, in some embodiments, the flowchart returnsto block 311, at which the newly selected path candidate is shown.

With respect to iteration through blocks 311, 312, and 314, reference isnow made to FIGS. 5A-5C, which schematically illustrate selection fromamong alternative vascular path options, according to some exemplaryembodiments of the present disclosure. FIG. 5A shows angiographic image500 illustrating a portion of a vascular tree 502 (a cardiacvasculature, in the example), including a currently defined rootposition 501. Contrast of angiographic image 500 is reduced in FIGS.5B-5F to make other elements more visible. In FIG. 5B, severalpreviously defined paths 507 are shown. Cursor 505 is shown hoveringdirectly over a vascular location, but the suggested path 509 insteadterminates some distance away from the cursor on another vascularlocation. In FIG. 5C, a user has indicated that the next candidatevascular path 511 should be shown, but this vascular path also fails toshow what the user intends to add to the accepted path list.

Finding that the path as intended is unavailable (per block 316), theuser optionally proceeds to block 320. With respect to this block,reference is now made to FIGS. 5D-5E, which schematically illustrate amethod of manually defining a vascular path option, according to someexemplary embodiments of the present disclosure. FIG. 5D shows aselection formed by a sequence of position indications (clicks made atdifferent positions of cursor 505, for example), with a line segmentdrawn between each indication. The user has made position indicationsgenerally along the intended vascular path. Optionally, root position501 is considered part of the position indications.

At FIG. 5E, the user has ended the sequence of position indications (forexample, with a double click). In some embodiments, this results in theoperations of block 322, in which the position indications are used tocreate a new path definition by a form of fitting between them, forexample, as described in relation to an active contour algorithm herein.The resulting vascular path is optionally added to the accepted pathlist; and the flowchart of FIG. 3A exits from block 112.

In some embodiments, another method of manually defining a vascular pathis used, for example, a variation of the Livewire technique, in which asegment position is optionally drawn live as between clicks definingwaypoints (“growing” to meet the cursor position from a previous anchorpoint), rather than all at once after all waypoints are initiallydefined.

Active Contour Method for Vascular Paths

In some embodiments of the disclosure, an active contour method, alsocommonly referred to as a “snake” dynamics method (Kaas et al. 1987) isimplemented. In this method, the behavior of the contour, c(s)=(x(s),y(s)), is governed by simulated intrinsic and extrinsic forces that areapplied on the contour. These forces are derived from a minimization ofan energy functional:

$\begin{matrix}{E = {\int{\left\lbrack {{E_{int}\left( {c(s)} \right)} + {E_{ext}\left( {c(s)} \right)}} \right\rbrack{ds}}}} & {{EQU}.\mspace{14mu} 1}\end{matrix}$

The internal energy term is:

$\begin{matrix}{{E_{int}\left( {c(s)} \right)} = {\frac{1}{2}\left( {{{\alpha(s)}\left( {\frac{dx}{ds} + \frac{dy}{ds}} \right)^{2}} + {{\beta(s)}\left( {\frac{d^{2}x}{ds^{2}} + \frac{d^{2}y}{ds^{2}}} \right)^{2}}} \right)}} & {{EQU}.\mspace{14mu} 2}\end{matrix}$

The external energy term E_(ext)(c(s)) is implicitly defined by a forcefield

$\begin{matrix}{{\overset{\rightarrow}{F}\left( {{x(s)},{y(s)}} \right)} = {{\gamma(s)}\left( {\frac{\partial{E_{ext}\left( {{x(s)},{y(s)}} \right)}}{\partial x},\frac{\partial{E_{ext}\left( {{x(s)},{y(s)}} \right)}}{\partial y}} \right)}} & {{EQU}.\mspace{14mu} 3}\end{matrix}$

As a force field, there is calculated in some embodiments the GGVF field(Xu and Prince, 1998) over a Frangi-filtered grayscale image (Frangi etal., 1998), although in general any force field can be used.

Active Contours for Initial Path Definition

When a vessel is first defined from the vascular path list 214, theinitial input may be taken from a concatenation of pixels in vascularskeleton segments that connect along a selected path. Optionally,coordinates actually used in the path are repositioned to fall atuniform intervals along the vascular path length (actualcenter-to-center distances of pixels themselves may be non-uniform dueto alternating movements in diagonal and cardinal directions). Then, asthe active contour method iterates, the vascular path positions aredrawn into new positions according to the various force and field termsused. Once the path anneals to a sufficiently stable configuration, theforce terms are set to zero, and the process stops.

Optionally, when a vessel is defined by sparsely provided anchor pointsas described, for example, in relation to FIGS. 5D-5E, the forces actingon the anchor points are set to zero (e.g., α(s), β(s), and γ(s) are setto zero), so that these points remain fixed. Points interpolated betweenthem (which may initially be provided as straight lines, by splineinterpolation, or by another method) are then subjected to non-zeroforces (e.g., α(s), β(s), and γ(s) are set to non-zero values) through anumber of iterations of numerically minimizing the energy functionals of(EQU. 2 and EQU. 3) until a sufficiently stable configuration isreached.

Active Contours for Path Editing

Reference is now made to FIG. 3B, which is a schematic flowchart of amethod of manually editing a vascular path, according to some exemplaryembodiments of the present disclosure. FIG. 3B represents an embodimentof the path editing operations of block 114 of FIG. 1. Reference is alsomade to FIGS. 5E-5F, which schematically illustrate manual editing of avascular path option, according to some exemplary embodiments of thepresent disclosure.

At block 340, in some embodiments, a user makes a position selectionalong length s of a displayed vascular path (which, in some embodiments,is a path from accepted path list 324). In some embodiments, theposition selection comprises a button press at some selected position ofa cursor, a gesture on a touch screen, or another input method. Insubsequent operations, the position selection is taken to define adragging point (e.g., dragging point 519A of FIG. 5E).

At block 342, in some embodiments, energy terms of α(s), β(s) and/orγ(s) are set to non-zero only for s within a restricted vicinity of thedragging point. Optionally, the external force field of EQU. 3 isdefined to comprise an attractor to the current cursor position, oranother position defined by further user interface actions by which theuser drags the dragging point to a new location.

At block 344, in some embodiments, the energy functionals of (EQu. 2 andEQU. 3) are numerically minimized, which may result in the displayedvascular path (e.g., path 517 of FIGS. 5E-5F) being displaced around thelocation of the dragging point.

At block 346, in some embodiments, display of the vascular path 517 isupdated.

At block 348, the system determines whether or not the draggingoperation is to be ended; for example, based on user input (e.g. releaseof a button or termination of a touch screen gesture). If not, at block350 in some embodiments, the user optionally drags the dragging point toa new location, for example, location 519B of FIG. 5F. Otherwise, theedited path 517 is updated in the accepted path list 324, and the flowchart ends (optionally, exiting block 114 of FIG. 1). It should beunderstood that in some embodiments, the accepted path list isadditionally or alternatively updated dynamically during the draggingoperation itself.

Vascular Path Definitions from a Plurality of Views

Reference is now made to FIG. 6, which schematically illustrates adisplay 620 of path definitions made concurrently on a plurality ofdifferent vascular image views 400, 500, 600, 610, according to someexemplary embodiments of the present disclosure.

In some embodiments, vascular image views 400, 500, 600, 610 eachpresent views from different angles of the same vasculature (e.g., aregion of a cardiac vasculature). Partially defined vascular path groups620, 622, and 624 are shown for views 400, 500, and 600. In someembodiments, homology among vascular image features (homologous imagefeatures represent the same anatomical location, optionally fromdifferent viewpoints) is established at least in part on the basis ofstructures identified in one or more of the vascular skeleton graph 216,the branch list 210, the path list 214, and or the accepted path list324. In some embodiments, the accepted path allows identification ofhomologous vascular features, optionally without detailed 3-Dinformation: for example, based on considerations of branch numbersand/or positions; root positions 611, 601, 501, 401; and/or relativevascular length. Optionally, basic 3-D information (for example, angleof view specified within about 45°, 90°, or another greater or lesseramount) helps in identifications by setting general relative orientationcorrectly, and/or identifying mirror-image views. In some embodiments,fuller 3-D information is available, for example, vascular centerlinepositions reconstructed based on stereoscopic projection from two ormore images into a common 3-D frame of reference (for example asdescribed in International Patent Publication No. WO2014/111930 to theapplicant, filed Jan. 15, 2014, the contents of which are incorporatedby reference herein in their entirety). Optionally, vascular pathsgenerated with respect to one image view are transformed so that theycan be superimposed on one or more alternative image views, based on thereconstructed 3-D frame of reference. In some embodiments, suchtransforms of vascular skeleton graphs 216, branch list 210, and/or pathlist 214 are performed, for example to assist in validating imageprocessing results. For example, missing segment portions in path and/orskeleton data from one image can be filled in from data available inviews taken from another image. Conversely, spurious segment portions inone image view are optionally identified, for example, based on theirabsence and/or lack of connection to the root position in other imageviews.

Reference is now made to FIG. 7, which is a schematic diagram ofsoftware modules and data structures 700 implemented in a system 702 forsemi-automated segmentation of vascular paths, according to someexemplary embodiments of the present disclosure.

Blocks 206, 210, 214, and 218, in some embodiments, comprise datastructures generated and/or used by the software modules of the system702. They correspond, for example, to the correspondingly numberedblocks of FIG. 2. Vascular images 701 are provided from an imagingdevice 750, which is optionally part of the system 702. Optionally,system 702 is stand-alone, or stand-apart from the imaging device 750,with the images are provided, for example, via a remote networkconnection.

Block 703, in some embodiments, comprises a vascular skeletonizer,configured for producing a skeletonized representation of vascularsegments (skeleton graph), for example as described in relation to block206 of FIG. 2.

Block 705, in some embodiments, comprises a path option manager. In someembodiments, path option manager 705 comprises software functions forimplementing operations of blocks 208, 212, and/or 216 of FIG. 2.

Editing module 710, in some embodiments, comprises sub-modules forhandling, for example, operations described in connection with FIGS. 1,3A, and/or 3B. Optionally, option presentation module 712 and/or manualpath definition module 716 implement block 112. Optionally, path editingmodule 714 implements block 114.

In some embodiments, system 702 additionally comprises display 760and/or input device(s) 770.

As used herein with reference to quantity or value, the term “about”means “within ±10% of”.

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean: “including but not limited to”.

The term “consisting of” means: “including and limited to”.

The term “consisting essentially of” means that the composition, methodor structure may include additional ingredients, steps and/or parts, butonly if the additional ingredients, steps and/or parts do not materiallyalter the basic and novel characteristics of the claimed composition,method or structure.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

The words “example” and “exemplary” are used herein to mean “serving asan example, instance or illustration”. Any embodiment described as an“example” or “exemplary” is not necessarily to be construed as preferredor advantageous over other embodiments and/or to exclude theincorporation of features from other embodiments.

The word “optionally” is used herein to mean “is provided in someembodiments and not provided in other embodiments”. Any particularembodiment of the disclosure may include a plurality of “optional”features except insofar as such features conflict.

As used herein the term “method” refers to manners, means, techniquesand procedures for accomplishing a given task including, but not limitedto, those manners, means, techniques and procedures either known to, orreadily developed from known manners, means, techniques and proceduresby practitioners of the chemical, pharmacological, biological,biochemical and medical arts.

As used herein, the term “treating” includes abrogating, substantiallyinhibiting, slowing or reversing the progression of a condition,substantially ameliorating clinical or aesthetical symptoms of acondition or substantially preventing the appearance of clinical oraesthetical symptoms of a condition.

Throughout this application, embodiments of this disclosure may bepresented with reference to a range format. It should be understood thatthe description in range format is merely for convenience and brevityand should not be construed as an inflexible limitation on the scope ofthe disclosure. Accordingly, the description of a range should beconsidered to have specifically disclosed all the possible subranges aswell as individual numerical values within that range. For example,description of a range such as “from 1 to 6” should be considered tohave specifically disclosed subranges such as “from 1 to 3”, “from 1 to4”, “from 1 to 5”, “from 2 to 4”, “from 2 to 6”, “from 3 to 6”, etc.; aswell as individual numbers within that range, for example, 1, 2, 3, 4,5, and 6. This applies regardless of the breadth of the range.

Whenever a numerical range is indicated herein (for example “10-15”, “10to 15”, or any pair of numbers linked by these another such rangeindication), it is meant to include any number (fractional or integral)within the indicated range limits, including the range limits, unlessthe context clearly dictates otherwise. The phrases“range/ranging/ranges between” a first indicate number and a secondindicate number and “range/ranging/ranges from” a first indicate number“to”, “up to”, “until” or “through” (or another such range-indicatingterm) a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numbers therebetween.

Although the example systems, methods, apparatuses, and/or computerprogram products have been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present disclosure. To the extent thatsection headings are used, they should not be construed as necessarilylimiting.

It is appreciated that certain features of the example systems, methods,apparatuses, and/or computer program products, which are, for clarity,described in the context of separate embodiments, may also be providedin combination in a single embodiment. Conversely, various features ofthe systems, methods, apparatuses, and/or computer program products,which are, for brevity, described in the context of a single embodiment,may also be provided separately or in any suitable subcombination or assuitable in any other described embodiment of the disclosure. Certainfeatures described in the context of various embodiments are not to beconsidered essential features of those embodiments, unless theembodiment is inoperative without those elements.

The invention is claimed as follows:
 1. An apparatus for editinglocations of vascular paths by enabling a user to move an erroneouslysegmented region of a vascular path into alignment with a blood vesseldepicted in a vascular image, the apparatus comprising: a memory devicestoring a vascular image depicting blood vessels; a user interface; anda processor communicatively coupled to the memory device and the userinterface, the processor configured to create the vascular pathcorresponding to one of the blood vessels by: estimating locations ofvascular segments, each estimated vascular segment defined within askeletonized representation of the vascular image by determining pixelsof the skeletonized representation of the vascular image that correspondto end points of the vascular segment, and determining a sequence ofpixel locations extending between the end points, connecting vesselsegments that have adjacent end points, determining an error among theestimated locations of the vascular segments based on at least one of anon-continuity of vascular width or radius between connected vascularsegments, a non-continuity of image intensity of the vascular imagebetween connected vascular segments, or a non-continuity of directionbetween connected vascular segments, causing the user interface todisplay a message indicative of the error, receiving from the userinterface an input for correction of the error, and adjusting at leastone of the vascular segments based on the input for the correction ofthe error.